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The  goal  of  this  project  is  to  examine  the  consequences  of  observer  motion  for  visual  func¬ 
tion.  The  research  has  focussed  on  two  issues:  One  issue  is  how  a  grossly  time-varying  retinal 
input  Q)ecaiise  of  eye,  head,  and  body  motion)  results  in  the  perception  of  a  continuous  and 
directionally  stable  visual  world.  A  second  issue  concerns  how  the  infonnation  in  successive 
views  is  related,  and  the  nature  of  the  visual  infonnation  retained  from  previous  views. 
Understanding  these  processes  is  important  for  a  wide  variety  of  visuo-motor  tasks. 


SUMMARY  OF  RESEARCH 

In  the  past  year  progress  has  been  made  on  the  following  six  projects. 

1)  The  role  of  the  visual  scene  and  eye  position  signals  in  visual  stability. 

In  collaboration  with  Jeff  Pelz,  I  have  devetoped  a  novel  technique,  using  the  perceived 
movement  of  afterimages  of  complex  scenes,  to  demonstrate  that  the  stability  of  die  visual 
scene  depends  on  the  nature  of  the  visual  context.  In  the  dark,  the  position  of  a  single  object 
is  computed  using  eye  position  information.  In  the  light,  however,  the  experiment  shows 
that  stationarity  is  attributed  to  the  visual  scene,  despite  eye  position  information  to  the  con¬ 
trary.  This  technique  is  relatively  easy  to  use  aixi  has  a  lot  of  potential  for  exploring  the  pro¬ 
perties  of  the  visu^  scene  which  determine  its  relative  weight  in  the  choice  of  a  stable  refer¬ 
ence  frame.  (This  technique  was  described  in  the  original  proposal  Experiment  3B.)  This 
work  was  reported  at  ARVO  in  1991  and  we  are  currently  preparing  a  manuscript. 

2)  The  role  of  attention  in  integrating  aaoss  saccades. 

This  work  is  in  collaboration  with  Joel  Lachter,  a  graduate  student  partially  funded  by  the 
project.  It  is  a  modified  version  of  Experiment  6  in  the  proposal.  In  an  earlier  experiment 
we  demonstrated  that  the  perception  of  form  can  be  achieved  whether  or  not  the  eye  is  sta¬ 
tionary,  and  that  very  precise  sp>atial  relationships  can  be  compnited  across  different  eye  p)osi- 
tions.  We  have  now  demonstrated  that  this  ability  requires  attention.  This  suggests  that 
only  a  sp>arse  representation  of  the  visual  scene  is  maintained  across  saccades.  (This  is  con¬ 
sistent  with  current  computational  'active  vision'  approaches.)  The  work  has  been  repxjrted  at 
ARVO  1991,  and  a  manuscript  is  in  progress. 

3)  Reference  frames  for  spatial  memory. 

Sp>atial  memory  plays  a  crucial  but  relatively  unexplored  role  in  much  of  our  motoric  interac¬ 
tion  with  the  world.  Previous  work  (described  in  Exp)eriment  10  in  the  proposal)  has  demon¬ 
strated  the  use  of  both  body  centered  and  object  centered  reference  frames  in  encoding  the 
p)ositions  of  objects.  In  this  expjeriment  Keith  Kam,  Per  Moeller  and  I  explored  whether 
object  centered  reference  frames  have  the  advantage  of  being  more  robust  across  a  series  of 


changes  in  eye  position.  Surprisingly,  both  forms  of  encoding  appeared  to  be  equally  robust. 
This  has  implications  for  the  source  and  precision  of  the  eye  position  information.  We  esti¬ 
mate  that  information  about  eye  position  in  a  head  centered  reference  frame  is  available  to 
the  visual  system  with  a  standiud  deviation  of  less  than  1.4  degrees.  This  work  was  reported 
at  ARVO  1^1  and  a  manuscript  has  been  accepted  for  publication  in  the  proceedings  of  the 
6th  European  Conference  on  Eye  Movements. 

4)  Hand-eye  coordination  during  complex  tasks 

Many  of  ^e  questions  tmder  investigation  in  this  grant  can  be  asked  most  naturally  in  the 
context  of  ongoing  'normal'  behavior.  In  collaboration  with  Dana  Ballard,  Steve  Whitehead, 
and  Feng  Li,  I  am  exploring  performance  of  a  complex  visuo-motor  task  with  the  goal  of 
building  a  computational  model  which  accurately  reflects  human  performance.  The  subject's 
task  is  to  copy  a  pattern  of  colored  blocks  on  a  computer  screen  using  the  mouse  to  move 
blocks  around  the  display.  Performance  appears  to  conform  to  a  relatively  stereotyped 
sequence  of  actions.  Computations  for  each  block  tend  to  be  kept  separate.  Although  some 
trials  reveal  the  use  of  visual  memory  from  previous  trials,  the  modal  response  pattern  points 
to  the  use  of  minimal  memory  in  performing  the  task.  Information  is  apparently  acquired 
only  just  before  it  is  needed.  Such  a  strategy  is  compatible  with  recent  computational  models 
of  robot  perfoimaixre  which  use  deictic  representations  to  selectively  index  the  parts  of  the 
scene  that  are  currently  relevant  to  the  task.  This  has  proved  vastly  more  efficient  than  con¬ 
ventional  models  which  exhaustively  represent  the  properties  and  locations  of  all  the  objects 
in  the  scene.  The  task  also  reveals  other  more  detailed  aspects  of  hand-eye  coordination,  not 
covered  in  prior  experimental  protocols.  We  observe  here  that  Ss  chose  to  make  nearly  simul¬ 
taneous  movements  to  disparate  hand  and  eye  targets  Such  movements  cannot  be  pro¬ 
grammed  by  a  single  central  motor  command,  as  has  been  suggested  from  observations  in 
simpler  paradigms.  We  are  currently  exploring  the  reference  frame  used  to  guide  the  eye 
movements  when  putting  down  the  blo^,  and  the  nature  of  the  memory  representations 
used  in  performing  the  task.  The  work  will  be  reported  at  ARVO  this  year,  and  a  manuscript 
with  preliminary  observations  is  in  progress. 

5)  Detectability  of  changes  during  saccades. 

In  a  more  formal  investigation  of  the  nature  of  the  visual  information  retained  from  previous 
views,  I  have  begun  work  on  a  project  designed  to  measure  the  detectability  of  changes  in 
shape,  color,  and  position  of  obj«rts  in  a  multiobject  display  when  the  changes  occur  during 
a  saccadic  eye  movement.  We  are  investigating  the  nature  of  the  visual  information 
preserved  from  the  immediately  preceding  fixation. 

6)  Short  tenn  visual  memory  of  complex  scenes. 

Jeff  Pelz  has  developed  a  technique  for  viewing  a  normal  scene  (reflective  surfaces)  through  a 
variable  sized  aperture  which  moves  with  the  eyes.  He  and  Greg  Zelinsky  (Brown  Univer¬ 
sity)  are  currently  working  on  a  project  which  examines  the  nature  of  the  information 
retained  from  a  short  view  of  a  scene.  This  new  version  of  the  aperture  viewing  technique 
has  a  range  of  interesting  applications  for  testing  the  richness  of  the  representation  of  previ¬ 
ously  and  currently  viewed  scenes. 


RELEVANT  PUBLICATIONS 


1.  Hayhoe,  M,  Lachter,  J.,  Feldman,  J.  (1991)  Integration  of  form  across  saccadic  eye  move¬ 
ments.  Perception,  20  393-402. 

2.  Hayhoe,  M.M.,  Lachter,  J.  &  Moeller,  P.  (1992)  Spatial  memory  and  integration  across  sac¬ 
cadic  eye  movements.  In  K.  Rayner  (Ed.),  Eye  Movements  &  Visual  Cognition.  Springer- 
Verlag.  (in  press) 

3.  Kam,  K,  Moeller,  P.,  and  Hayhoe,  M.  Precision  of  the  eye  position  signal.  (1992)  To 
appear  in  Van  Rensbergen,  J.  A  d'Ydewalle,  G.  (Eds.),  Studies  in  Visual  Attention.  North  Hol- 
latid. 

4.  Ballard,  D.,  Hayhoe,  M.,  A  Whitehead,  S.  (1992)  Hand-Eye  coordination  during  sequential 
tasks.  To  appear  in  Proc  Roy  Soc  B. 


MANUSCRIPTS  IN  PREPARATION 

1.  Moeller,  P,,  Hayhoe,  M.,  Ballard,  D.,  Albano,  J.  Saccades  to  remembered  visual  targets  and 
the  perception  of  spatial  position,  (in  preparation). 

2.  Lachter,  J.,  Hayhoe,  M.  A  Feldman,  J.  Capacity  limits  in  the  integration  of  information 
across  saccades.  (in  preparation) 

3.  Pelz,  J.  A  Hayhoe,  M.  Influence  of  the  visual  scene  in  space  constancy,  (in  preparation) 


PRESENTATIONS  AT  SOENTIHC  MEETINGS 

1991,  K.  Kam,  P.  Moeller,  and  M.  Hayhoe,  "Eye  Movements  to  Remembered  Targets;  Dismp- 
tion  by  Intervening  Saccades."  Paper  present^  at  meeting  of  the  Association  for  Research  in 
Vision  and  Ophthalmology,  Sarasota,  Florida. 

1991,  J.  B.  Pelz  aiKi  M.  M.  Hayhoe,  "Influence  of  the  Visual  Scene  and  Eye  Position  Signals  in 
Space  Constancy."  Paper  presented  at  meeting  of  the  Association  for  Research  in  Vision  and 
Ophthalmology,  Sarasota,  Florida. 

1991,  J.  Lachter,  M.  Hayhoe,  and  J.  Feldman,  "Capacity  Limitations  in  the  Integration  of 
Information  Across  Saccades."  Paper  presented  at  meeting  of  the  Association  for  Research  in 
Vision  and  Ophthalmology,  Sarasota,  Florida. 


PERSONNEL 

In  the  past  year  six  graduate  students  have  participated  in  this  project.  They  are  supported 
by  a  combination  of  funds  from  this  grant,  an  NIH  training  grant,  and  University  funds.  The 
students  are  Joel  Lachter,  Per  Moeller,  Keith  Kam,  Jeff  Pelz,  Brady  Duga,  and  Feng  Li.  In  the 


coining  year  Lachter,  Moeller,  Kam,  and  Pelz  will  continue  to  work  on  the  project.  In  addi¬ 
tion  to  this,  a  graduate  student  from  Brown  University,  Greg  Zelinsky,  visited  for  two 
months  under  our  Summer  Fellowship  program.  Steve  Whitehead  joined  the  project  in 
November  as  a  post  doctoral  fellow  after  completing  a  degree  in  Computer  Science  at  the 
University  of  Rochester  with  Dana  Ballard. 


EQUIPMENT 

A  MacIIfX,  Super  Mac  large  screen  monitor,  24  bit  color  board,  and  cartridge  drive  have  been 
piurhased.  This  is  being  used  in  conjunction  with  one  of  the  DPI  trackers  for  experiments 
where  a  large  display  is  important. 


